DOCUMENT RESUME 



ED 034 971 



AL 002 121 



AUTHOR 

TITLE 

SPONS AGENCY 
PUB date 

note 

AVAILABLE FROM 



Briere, Eugene J. 

English Language Testing Project for the Bureau of 
Indian Affairs, 

Bureau of Indian Affairs (Dept, of Interior) , 

Washington, D.C. 

r691 

15p. 

Author, English Language Testing Project, 2140 
Westwood Bivd., Los Angeles, California 90024 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



EDRS Price MF~S0.2 5 HC-S0,.,85, .. 

^^'American IndiaTis,- Auditory Visual Tests, Elementary 
Education, ^English (Second Language) , ^Equivalency 
Tests, Group Noru;s, Language Proficiency, Language 
Shills, Listening Comprehension, Measurement 
Instruments, Minority Group Children, Standardized 
Tests, ^Testing, ^Testing Programs 
^Bureau of Indian Affairs Schools 



ABSTRACT 



This brochure contains a summary of an English 
language testing project being developed by the University of 
Southern California under the aegis of the Bureau of Indian Affairs 
(BIA) . Previous attempts (Stanford Achievement Test, California 
Achievement Test) have failed to test American Indian children’s 
performance in reading, vocabulary, and language because these tests 
are not designed to measure linguistic skills. The norms and 
standards of such tests are derived from populations far different 
from the American Indian children in BIA schools, many of whom are 
either non-English speaking, or seriously deficient in English. The 
major objectives of this Project are to develop a battery of tests 
consisting of three parts: (1) three oral-aural technigues to measure 

proficiency in perception and production of spoken English, (2) two 
technigues for testing proficiency in listening comprehension, and 
(3) a paper and pencil test designed to measure the level of 
proficiency in written English grammar. Although not specifically 
within the scope of this three-year project, it is felt that the 
testing instruments being developed will be of great use in assessing 
proficiency in English for other non-English speaking groups. Direct 
inquiries to Author, or to Assistant Commissioner for Education, BTA, 
1951 Constitution Ave. NW, Washington, D.C. 20242. (AMM) 
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This brochure contains a suimnary of a project being developed by the 
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University of Southern California, Dr. Eugene Briere, Professor. of 
Linguistics, Those desiring detailed information rcgardiT?g„the 



report may direct their inquiry to Dr. Briere or to; 

Assistant Commissioner for Education 
Bureau of Indian Affairs 
1951 Constitution Avenue, N.W. 
Washington, D. C. 20242 



Attention: Curriculum Division 
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English Language Testing Project 

2140 Westwood Blvd., Los Angeles, California 90024 



Director Dr. Eugene J. Briere 
Associate Professor of Linguistics 
University of Southern California 



□ INTRODUCTION 



There is an ever increasing need for a solution to the problem of non- 
English speakers, specifically American Indian children, to perform satis- 
factorily in school. From Alaska to the southern tip of Florida, many of the 
American Indian children attending Bureau of Indian Affairs’ schools are 
non-English speaking, William H. Kelly states in his 1967 paper. “Current 
Research on American Indian Education: A Critical Review of Selected 
Ongoing Studies,” that “most Indian students either do not speak Eng- 
lish before entering school or are seriously deficient in their knowledge 
of English.” This deficiency in English becomes a major disadvantage in 
Bureau of Indian Affairs' schools where the American Indian child must 
study and respond in English. Consequently, it is extremely important to 
be able to measure the level of proficiency in English for each American 
Indian child. Studies by Peterson (1948), Anderson (1953), and Coombs 
(1958) showed that the performance of the American Indian child on 
standard tests of reading, vocabulary and language was generally lower 
than that of non-Indians. 

All previous attempts, i.e. Stanford Achievement Test, California 
Achievement Test, failed in this respect because they are not designed to 
measure linguistic skills, and the norms and standards are derived from 
populations far different from the American Indian children. Furthermore, 
the study of Havighurst (1957) strongly indicated that the types of tests 
which require facility in English tend to discriminate against the American 
Indian child who does not learn English at home. As a result, any attempt 
to use existing standardized tests to measure the linguistic skills of the 
American Indian are biased. 



ANY ATTEMPTS TO USE EXISTING STANDARDIZED 
TESTS TO MEASURE THE LINGUISTIC SKILLS OF 
THE AMERICAN INDIAN ARE BIASED. 



Clearly, the American Indian child's deficiency in English will prove to 
be a great disadvantage not only to his overall achievement test perform- 
ance but also to his classroom performance since all subjects require a 
fairly sound conimand of English. This Indian child often needs special 
training in English as a second language. It is therefore important to 
identify the child attending Bureau of Indian Affairs’ schools who require 
special training versus the one who does not. Hopkins (1957) in reviewing 
the type of testing which has been done with North American Indians, 
suggests that there is a need for a testing instrument which will identify 
the child who needs ESL training. 



In general, the Bureau of Indian Affairs has been interested in the 
investigation of the language problems and is interested in developing 
meaningful innovations in methods and materials that are based on 
empirical evidence. Aware; of the problems existing because of the use of 
English in Bureau of Indian Affairs’ schools, Charles N. Zellers, Assistant 
Commissioner of Education for the Bureau of Indiab Affairs, has called 
for an instrument to assist teachers and administrators in identifying 
language problems and finding effective solutions. 

□ THE ENGLISH LANGUAGE TESTING PROJECT 

The purpose of the English Language Testing Project, directed by Dr. 
Eugene J. Brifere, Associate Professor of Linguistics at the University of 
Southern California, is to develop a set of testing instruments that will 
determine the level of proficiency in English for the American Indian child 
so he can be placed in the proper level of instruction in school which will 
give hirn the greatest opportunity for academic success, in aodition to 
measuring the active skills of speaking and writing, passive skills such as 
listening comprehension aire being tested. Six different techniques are 
being developed to measure the knowledge of specific grammatical cate- 
gories determined by psycholinguists as being necessary for having a 
fluent command of English. 

When the final forms of the various tests In the battery are completed, 
norms and standards for the various populations will be determined. The 
tests should not only place the child along a continuum of proficiency in 
English but will also clearly indicate the children who must have varying 
degrees of training in English as a second language. 

In addition the battery of tests will provide diagnostic information con- 
cerning the specific English language needs of each child for teachers and 
administrators. This will be a definite aid in determining the methods and 
materials to be used for each specific group. 

The major objectives of this project are to develop a battery of tests, 
consisting of three parts, specifically designed to measure the level of 
English proficiency of the American Indian child. One part of the test bat- 
tery will consist of three oral/aural techniques designed to measure the 
level of proficiency in perception and production of spoken English. A 
second part of the battery will contain two techniques for testing pro- 
ficiency in listening comprehension. The third part of the battery will be a 
paper and pencil test designed to measure the level of proficiency in 
written English grammar. 

These general objectives v/ill require a three-year project, (the first year 

of which has already been completed) which will be phased in the follow- 
ing manner: 

a. The first year was devoted to planning and developing pre-tests and 
techniques. 

b. The second year will be spent in developing final form instruments 
based on the data gathered in the first and second years. 

c. The third year will be devoted to standardizing and normalizing the 
tests specifically for the American Indian child. 



THE PROJECT IS DEVELOPING A BATTERY OF 
TESTS SPECIFICALLY DESIGNED TO MEASURE 
THE LEVEL OF ENGLISH PROFICIENCY OF THE 
AMERICAN INDIAN CHILD. 



□ DESCRIPTION OF TESTS 

The following different test forms are being developed to meet the 
objectives just described. 

A written multiple choice test consisting of two different parallel forms 
is designed to test 53 basic grammatical categories. The items all have 
three choices and are of two types. In one type the stem is in the form of a 
question and the choices are in the form of answers. In the second type, 
the stem is in the form of an incomplete sentence (i.e. a sentence with a 
blank in it) and the choices are in the form of words or phrases to be filled 
in. Both types require knowledge of the specific grammatical category 
being tested in order to answer the item correctly. 

Example: Type 1 : What color is the pencil? 

a) It’s yellow. 

b) The yellow pencil. 

c) Yes, it’s a pencil. 

Type 2: John isn’t as as Rose. 

a) tallest. 

b) tall. 

c) taller. 

Both listening comprehension tests consist of taped stimuli that the 
children hear and some form of non*oral response. One test involves a 
single stimulus sentence per item and three pictures as the response 
choices. The child hears the stimulus sentence, then decides which of 
the three pictures best illustrates that sentence by marking an “X” 
through that picture. 

Example: The child hears the sentence “The ball’s under the box.’’ 
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“X" through picture 3. 





The other listening comprehension test involves short conversations 
between two people. For each item, the child hears a taped conversation. 
He then hears a question about the conversation and three possible 
answers. Both the question and the answers appear in written form in 
his answer booklet. The reason for taping the questions and answers is to 
reduce a child's chances of missing a question due to poor reading ability. 
Having heard the question and possible answers, the child marks his 
choice in the answer bool'Jet. 

Example: Conversation: 

A. “Why don’t you take the broom and sweep up over there 
John . . . under the desk?" 

B. “O.K. Do I get to erase the blackboard too?” 

C. “Sure.” 

Question: 

Where did this conversation probably take place? 

a) In a classroom. ^ 

b) In a market. 

c) In a movie theater. 



The battery of oral tests is designed to determine the child’s speaking 
ability. Three types of tests are involved. They are; 

1) Repetition: the child simply repeats a sentence that the examiner says 
first. 

Example: The dog that got lost came back today. 

2) Transformation ; The child changes a given sentence to a related struc- 
ture. For example, he is asked to make a positive sentence negative. 
Example: Examiner: “He knows how to play that game.’’ 

Child: “He doesn’t know how to play that game.’’ 

3) Picture Description: The child is shown a set of four related pictures 
and then shown a picture that is exactly like one of the four. He must 
point to the correct picture and then tell how that picture is different 
from the rest. 
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All responses in this oral battery are tape recorded and later analyzed 
by our staff. 

The repetition and transformation tests are presently judged on the 
basis of accuracy of reply. On the picture description test, the responses 
are judged on the basis of grammaticality, appropriateness of vocabulary, 
and ability to distinguish the target picture from the other three. In the 
case of the listen! ng tests and written muitipie choice tests, responses are 
either right or wrong. 

Revision in the test materiais are made after the test results are 
analyzed after each administration. Individual test results and other per- 
tinent data, su:h as teacher grades and scores on standardized tests, like 
the California Achievement Test, are analyzed and correlated through the 
use of an IBM computer. 



The experimental tests are being administered to five sample American 
Indian populations in the United States. These populations comprise five 
different language groups: Choctaw, Eskimo, Hopi, Navajo and Sioux. 
Before the tests are given to the Indian children, they are pre-tested with 
groups of Mexican-American children and native speakers of American 
English at comparable grade levels of the American Indian group. The 
tests, revised on the basis of these preliminary data, are then adminis- 
tered to the Indian children. 

Throughout the months of September and October 1968, 500 children 
from grades three through seven were tested on the oral tests and 735 
were tested on the written tests. In April and May of 1969, revised experi- 
mental tests were administered to a total of 720 children from the same 
five tribes. This represents a total sample population of 1477 American 
Indian children tested in grades three through seven. In addition approxi- 
mately 920 native speakers of American English and native speakers of 
Spanish were tested in our pre-test samples. 

□ TESTING RESULTS 

One of the most obvious discoveries was that children who attend 
Bureau of Indian Affairs' schools which have an English as a second 
language (ESL) program operating in the very beginning grade levels 
score significantly higher than children who attend Bureau of Indian 
Affairs' schools where there is a very limited ESL program or an ESL pro- 
gram that does not start until grade nine or later. (Three charts below). 

On the basis of correlations within specific parts of the battery and 
across different parts of the battery, it is obvious that our different instru- 
ments are indeed measuring different skills. 




In each of the charts shown. Tribe “A" has had extremely limited ESL 
training; Tribe “B" has had very limited ESL training; and Tribe “C" 
has had ESL training starting from the very beginning grades. The scores 
made by Tribe “C" were statistically significantly better than the other 
two tribes shown on these charts at a significance level which is better 
than .008. 










MEAN NUMBER OF CORRECT RESPONSES MADE ON 
WRITTEN FORM IM A 



MEAN NUMBER OF CORRECT RESPONSES MADE DN 
REPETITIDN FDRM RT B 




TEACHERS^ INTUITIVE RATINGS OF ENGLISH 
PROFICIENCY DO NOT CORRELATE WELL WITH 
ACTUAL TEST SCORES. 



It is interesting to note that the intuitive ratings made by teachers of 
the children's proficiency in English do not correlate very well with scores 
made on our tests or with scores made on standardized tests. In other 
words, the tests being developed by the English Language Testing Project 
will help avoid errors in placement that are currently frequently made on 
impressionistic judgements rather than on empirical evidence. 

By the end of the third year, the diagnostic information concerning 
specific grammatical categories that will be available to teachers and 
administrators should prove invaluable in helping to write or select proper 
materials and more efficient teaching methods for each specific group. 
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□ CONCLUSION 



This project is in the process of developing a set of instruments to 
measure the level of proficiency in English of the American Indian child 
attending Bureau of Indian Affairs' school. 

The first tentative experimental results have been most gratifying and 
on the advice of our board of consultants (representing the disciplines of 
tests and measurements, linguistics, psychology, computer science, and 
cross cultural education), we feel that we can develop a battery of tests 
that will be extremely useful in planning for the total education of the 
Amerindian school child. 

Although not specifically within the scope of this project, it is felt that 
the testing instruments being developed will be of great use in assessing 
proficiency in English for other non-English speaking groups and dis- 
advantaged groups in the United States. 




THE FINAL BATTERY OF TESTS WILL BE 
EXTREMELY USEFUL IN PLANNING FOR THE 
TOTAL EDUCATION OF THE AMERINDIAN 
SCHOOL CHILD. 
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The principal investigator has had experience in teaching English 
as a second language, psycholinguistics, linguistics and language 
testing. His recent work for the last five years has been in the field 
of testing languages at the phonological and morpho-syntactic 
levels. A brief bibliography follows: 

Eugene J. Briere, Director. Ph.D. in language and Language 
Learning. University of Washington, 1964. Associate Professor of 
Linguistics, University of Southern California: teaches courses in 
general linguistics and graduate seminars in experimental linguis- 
tics, language testing, psycholinguistics and TESOL methodology. 
Member of the editorial boards of Language, the TESOL Quarterly, 
and Modern Language Journal, former Chairman of the General 
Linguistics Section of the MLA. Author of several articles dealing 
with language acquisition and language testing appearing ir- Lan- 
guage, Language Learning, International Journal of American 
Linguistics, International Review of Applied Linguistics and 
English Teaching Forum. His book, A Psycholinguistic Investiga- 
tion of Phonological Interference, Mouton and Co., January 1968, 
deals with problems in phonological testing. He has just completed 
a project which developed achievement tests in English, based on 
the Fries-Rojas materials, for the Navajo Area Agency. 
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